Convergent and Divergent Validity Documentation Based on Relations Among Content Areas

To examine convergent and divergent validity evidence, (a) correlations were were conducted to examine the relation between scale scores in the different content areas (ELA with Math, ELA with Science, and Math with Science), and (b) Chi-squared tests of independence were conducted to examine whether performance level scores (i.e., Advanced, Proficient, Emerging, of Novice) were independent across content areas at the p = .01 significance level. Note that the p-values were computed for a Monte Carlo test (Hope, 1968) with 2,000 replicates.

6.1 Correlations Among Content Areas

Correlations for ELA with Math ranged from 0.29 to 0.71, correlations for ELA with Science ranged from 0.61 to 0.82, and correlations for Math with Science ranged from 0.32 to 0.61. Please see Table 6.1.1. Across grades, correlations are almost always higher for Tier 1 than Tier 2 (except Grades 8 ELA with Math, and all Grade 11 correlations). Across Grades 4, 8 and 11, correlations for ELA with Science were higher than correlations for Math with Science.

Table 6.1.1. Correlations between Content Areas by Grade and Tier
Grade Tier ELA with Math ELA with Science Math with Science
3 1 .51 – –
3 2 .51 – –
4 1 .69 .82 .61
4 2 .42 .70 .40
5 1 .71 – –
5 2 .39 – –
6 1 .69 – –
6 2 .35 – –
7 1 .71 – –
7 2 .36 – –
8 1 .36 .77 .37
8 2 .39 .66 .32
11 2 .29 .61 .37
11 1 .41 .80 .40

6.2 Chi-squared Tests of Independence for Performance Levels Among Content Areas

Across grades and tiers, all Chi-squared tests of independence were statistically significant, except Grade 4, Tier 2 ELA - Science. Thus, in general, performance level scores in one content area were not independent of performance level scores in another content area. Please see tables….

Table 6.2.1. ELA - Math Performance Levels Chi-squared Test of Independence by Grade and Tier
Grade Tier Chi-Square p-value
3 1 258.83 < .01
3 2 89.37 < .01
4 1 831.69 < .01
4 2 113.63 < .01
5 1 533.11 < .01
5 2 145.75 < .01
6 1 572.22 < .01
6 2 170.04 < .01
7 1 513.27 < .01
7 2 181.48 < .01
8 1 223.29 < .01
8 2 161.59 < .01
11 2 95.57 < .01
11 1 123.39 < .01
Table 6.2.2. ELA - Math Performance Matrix for Grade 3 Tier 1
Math
ELA Advanced Proficient Novice Missing
Advanced 79 0 0 9
Proficient 448 7 0 28
Emerging 326 167 1 59
Novice 198 4 1 17
Table 6.2.3. ELA - Math Performance Matrix for Grade 3 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 52 39 0 9 6
Proficient 6 28 0 11 7
Emerging 0 0 2 2 2
Novice 2 10 5 15 5
Table 6.2.4. ELA - Math Performance Matrix for Grade 4 Tier 1
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 151 221 6 21 29
Proficient 32 152 10 70 33
Emerging 0 26 229 83 58
Novice 5 64 44 80 17
Table 6.2.5. ELA - Math Performance Matrix for Grade 4 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 76 48 0 7 10
Proficient 22 57 6 23 14
Emerging 0 0 3 1 1
Novice 0 1 3 3 1
Table 6.2.6. ELA - Math Performance Matrix for Grade 5 Tier 1
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 34 18 0 2 4
Proficient 114 149 21 63 43
Emerging 3 12 217 80 41
Novice 12 76 81 83 24
Table 6.2.7. ELA - Math Performance Matrix for Grade 5 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 57 58 3 16 8
Proficient 26 73 35 69 15
Emerging 0 2 17 8 2
Novice 7 29 40 27 8
Table 6.2.8. ELA - Math Performance Matrix for Grade 6 Tier 1
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 110 134 11 50 47
Proficient 8 51 29 46 13
Emerging 1 15 264 29 36
Novice 2 32 54 37 16
Table 6.2.9. ELA - Math Performance Matrix for Grade 6 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 83 35 0 6 10
Proficient 127 118 13 63 29
Emerging 0 1 9 2 2
Novice 10 28 10 24 6
Table 6.2.10. ELA - Math Performance Matrix for Grade 7 Tier 1
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 95 62 4 35 31
Proficient 26 60 36 63 29
Emerging 1 4 186 29 17
Novice 2 17 90 70 21
Table 6.2.11. ELA - Math Performance Matrix for Grade 7 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 187 130 3 50 34
Proficient 53 97 4 60 19
Emerging 0 3 6 18 3
Novice 8 24 2 42 8
Table 6.2.12. ELA - Math Performance Matrix for Grade 8 Tier 1
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 130 10 3 4 20
Proficient 147 16 7 1 20
Emerging 25 94 2 0 14
Novice 152 71 5 0 37
Table 6.2.13. ELA - Math Performance Matrix for Grade 8 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 27 135 11 31 24
Proficient 22 110 28 72 31
Emerging 14 10 42 20 11
Novice 18 34 29 62 12
Table 6.2.14. ELA - Math Performance Matrix for Grade 11 Tier 2
Math
ELA Advanced Proficient Emerging Novice Missing
Advanced 231 45 0 15 14
Proficient 45 30 0 16 7
Emerging 10 12 1 16 2
Novice 11 7 0 6 4
Table 6.2.15. ELA - Math Performance Matrix for Grade 11 Tier 1
Math
ELA Advanced Proficient Novice Missing
Advanced 46 123 1 5
Proficient 6 72 0 4
Emerging 1 266 0 42
Novice 0 65 0 3
Table 6.2.16. ELA - Science Performance Levels Chi-squared Test of Independence by Grade and Tier
Grade Tier Chi-Square p-value
4 1 421.51 < .01
4 2 59.14 < .01
8 1 388.10 < .01
8 2 201.48 < .01
11 2 62.73 < .01
11 1 182.21 < .01
Table 6.2.17. ELA - Science Performance Matrix for Grade 4 Tier 1
Science
ELA Proficient Emerging Novice Missing
Advanced 225 1 161 41
Proficient 96 3 152 46
Emerging 12 99 201 84
Novice 48 4 131 27
Table 6.2.18. ELA - Science Performance Matrix for Grade 4 Tier 2
Science
ELA Proficient Emerging Novice Missing
Advanced 105 0 22 14
Proficient 44 1 60 17
Emerging 0 0 4 1
Novice 1 0 7 0
Table 6.2.19. ELA - Science Performance Matrix for Grade 8 Tier 1
Science
ELA Advanced Proficient Emerging Novice Missing
Advanced 9 70 1 63 24
Proficient 0 42 9 107 33
Emerging 0 0 101 11 23
Novice 0 15 98 116 36
Table 6.2.20. ELA - Science Performance Matrix for Grade 8 Tier 2
Science
ELA Advanced Proficient Emerging Novice Missing
Advanced 35 142 3 22 26
Proficient 9 156 4 55 39
Emerging 0 21 10 49 17
Novice 0 50 4 84 17
Table 6.2.21. ELA - Science Performance Matrix for Grade 11 Tier 2
Science
ELA Advanced Proficient Novice Missing
Advanced 32 215 39 19
Proficient 0 47 41 10
Emerging 0 20 16 5
Novice 1 15 8 4
Table 6.2.22. ELA - Science Performance Matrix for Grade 11 Tier 1
Science
ELA Advanced Proficient Emerging Novice Missing
Advanced 12 116 0 40 7
Proficient 0 45 0 31 6
Emerging 0 70 52 131 56
Novice 0 27 1 37 3
Table 6.2.23. Math - Science Performance Levels Chi-squared Test of Independence by Grade and Tier
Grade Tier Chi-Square p-value
4 1 893.76 < .01
4 2 137.03 < .01
8 1 479.75 < .01
8 2 296.37 < .01
11 2 197.10 < .01
11 1 376.10 < .01
Table 6.2.24. Math - Science Performance Matrix for Grade 4 Tier 1
Science
Math Proficient Emerging Novice Missing
Advanced 129 1 48 10
Proficient 164 3 261 35
Emerging 14 98 149 28
Novice 61 4 162 27
Missing 13 1 25 98
Table 6.2.25. Math - Science Performance Matrix for Grade 4 Tier 2
Science
Math Proficient Emerging Novice Missing
Advanced 81 0 14 3
Proficient 49 0 47 10
Emerging 3 0 9 0
Novice 11 1 20 2
Missing 6 0 3 17
Table 6.2.26. Math - Science Performance Matrix for Grade 8 Tier 1
Science
Math Advanced Proficient Emerging Novice Missing
Advanced 8 109 73 231 33
Proficient 0 10 128 38 15
Emerging 0 1 2 13 1
Novice 0 3 0 2 0
Missing 1 4 6 13 67
Table 6.2.27. Math - Science Performance Matrix for Grade 8 Tier 2
Science
Math Advanced Proficient Emerging Novice Missing
Advanced 11 34 4 29 3
Proficient 28 187 2 49 23
Emerging 1 40 7 54 8
Novice 3 91 3 73 15
Missing 1 17 5 5 50
Table 6.2.28. Math - Science Performance Matrix for Grade 11 Tier 2
Science
Math Advanced Proficient Novice Missing
Advanced 32 210 45 10
Proficient 1 53 34 6
Emerging 0 0 1 0
Novice 0 28 22 3
Missing 0 6 2 19
Table 6.2.29. Math - Science Performance Matrix for Grade 11 Tier 1
Science
Math Advanced Proficient Emerging Novice Missing
Advanced 9 38 0 5 1
Proficient 3 216 50 229 28
Novice 0 0 0 1 0
Missing 0 4 3 4 43